The pitfall in comparing benchmarks using hardware performance counters

Understanding the behavior of emerging workloads is important for designing next generation microprocessors. For addressing this issue, computer architects and performance analysts build benchmark suites of new application domains and compare the behavioral characteristics of these benchmark suites against well-known benchmark suites. Current practice typically compares workloads based on microarchitecture-dependent characteristics measured using hardware performance counters while running these workloads. There is one pitfall though with comparing benchmarks using microarchitecturedependent characteristics, namely that completely different inherent program behavior may yield similar microarchitecture-dependent behavior. We illustrate this by comparing the distance between benchmarks using both hardware performance counter metrics and microarchitecture-independent characteristics.